Learning Software Organizations and Agile Software Development: Complementary or Contradictory Concepts?

The LSO Workshop series has focused on technical, organizational, and social solutions to problems of learning from past experiences and codifying the resulting best practices so they can be systematically used in subsequent software development efforts. The theme of the Fourth Workshop on Learning Software Organizations (LSO), held in Chicago, Illinois in August of 2002 was "Balancing Agile Processes and Long-Term Learning in Software Organizations". In this workshop, we wanted to explore the issues involved in managing knowledge in dynamic domains requiring significant levels of improvisational change within each repetition of the process [1]. Thus, it seemed appropriate to collocate with the XP Agile Universe conference, which focuses on Agile development methods that require significant levels of flexibility in the development process. In this light, the challenge for the LSO community becomes one of understanding how the lessons learned and knowledge constructed in past projects can be brought to bear on subsequent software development efforts. The LSO community must therefore balance the capture and dissemination of knowledge with necessary flexibility that enhances the ability of an organization to quickly adapt its processes to new technologies and market pressures [2]. The Agile community, on the other hand, is faced with the opposite problem. Contrary to popular belief, the Agile community is not against documentation, but is prudently suspicious of over-documentation, especially when requirements are volatile [3]. But this may lead to situations where institutional knowledge is lost when people move on, are moved to new projects, or the size of an organization becomes big enough to impede the low of knowledge, making it difficult to improve or avoid repeating failures. The agile approach of knowledge sharing by face-to-face communication also limits learning to occur between independent teams in a development organization. So the problems facing these communities seem complementary. At its core, the desire is to design work practices that balance the desire for innovation with knowledge of past experiences and best practices that will enable improvement in quality, productivity, and etc. This tension between past knowledge and innovation is particularly acute in the software industry, which involves the development of a highly variable product that dictates the need for continuous process adjustments.